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Ke ntucky educato rs With focus, porsienents, ghee a collective Corinne 
: to results-driven accountability, teachers in four districts 
are defying the have improved mathematics outcomes for students 
with disabilities in general educations settings. Their 
odds. 


efforts have also improved outcomes for Black students 
who typically fall behind their peers in mathematics 
achievement (Berry et al., 2014; National Center for 
Learning Disabilities, 2020). This brief presents the results 
from an evaluation of four school districts’ efforts to 
improve mathematics achievement in Kentucky. 


The purpose of the study was to investigate whether the 
districts that received ongoing training and coaching to 
use the Active Implementation Frameworks (https://nirn. 
fpg.unc.edu/ai-hub) improved mathematics outcomes. The 
Active Implementation Frameworks represent five essential 
approaches for effective implementation of any evidence- 
based program or practice. One of the frameworks is a 
clearly defined and operationalized usable innovation, 

or any evidence-based practice that is easily measured 
for adherence to the program design. The subject of 
Kentucky’s usable innovation is mathematics and use of 
the 8 mathematical teaching practices developed by the 
National Council of Teachers of Mathematics (NCTM, 
2014). 
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The evaluation focused on priority populations, specifically 
students with disabilities and Black students, and 
compared their outcomes to a carefully matched district 
that did not receive the same level of support. Results 
show that the four districts that received ongoing support 
improved mathematics outcomes for students with 
disabilities and students who are Black compared to the 
matched district. 


Active 
Implementation 


In 2014, the Kentucky Department of Education partnered 
with the State Implementation and Scaling-up of Evidence- 
Based Practices (SISEP) Center to effectively implement 
the State Systemic Improvement Plan. They aimed 
primarily to improve student mathematics outcomes 

for students with disabilities. Participating leaders and 
educators believed that the quality of teachers’ instructional 
practice in the classroom was the most significant factor 

in teaching mathematics for closing long-standing deficits 
and disparities in student outcomes (Harper, 2019; Hattie, 
2003; NCTM, 2020). 
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The Transformation Zone 
is a vertical slice of the 
system, small enough 

to be manageable, large 
enough to represent equity 
of context and need, and 
large enough to disturb the 
system for trial, learning, 
and scalability. 


To achieve their aims, Kentucky stakeholders with diverse 
perspectives and roles created a quality standard for 
mathematics instruction delivered in the general education 
setting. Then Kentucky educators with specialized 
expertise created training, coaching, and data systems for 
use in participating districts and their schools. Teachers 
received support to use non-evaluative observation data 
in aggregate, without teacher identifiers, to set schoolwide 
goals for instructional improvement. Teams at the school, 
district, region, and state delivered the supports and 
resources teachers required to continuously improve their 
instructional practice and meet their schoolwide goals for 
improvement. 


To ensure equitable access to effective instruction by every 
student in every classroom, leaders and educators focused 
on removing the variability in the support systems available 
for teachers and school staff. Districts represented a 
variety of contexts (large urban to small rural) to increase 
the chance that the model could scale to additional districts 
once fully developed and tested in the first iteration of 
Kentucky’s Transformation Zone. This brief presents 

initial findings from the State Systemic Improvement Plan 
initiative in five districts and 33 schools. Districts and their 
schools who received intensive support are referred to 

as Transformation Zone or TZ schools. The comparison 
district and their schools who did not receive intensive 
support are referred to as non-TZ. 
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St ate System Cc To achieve effective and durable change, Kentucky 


focused on effectively implementing the State Systemic 


Improvement Plan Improvement Plan to meet the Kentucky Department of 


Education’s State Identified Measurable Result: 


To increase the percentage of students 
with disabilities performing at or above 
proficient in middle school math, 
specifically at the 8th grade level, 

with emphasis on reducing novice 
performance, by providing professional 
learning, technical assistance and 
support to elementary and middle 
school teachers around implementing, 
scaling, and sustaining evidence- 
based practices in math. 
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Results 


aA 


AIS 


To evaluate whether the Kentucky Department of 
Education met its goals for students with disabilities, 

this investigation presents preliminary results across 
four school years. Data collection began in the 2015- 

16 school year, and initial implementation of the State 
Systemic Improvement Plan began in the 2016-17 school 
year. Figures A, B, and C display the percent of students 
in novice, apprentice, and proficient and distinguished 
categories for math from the Kentucky Performance 
Rating for Educational Progress (K-PREP). Tables 1-9 
in the Appendix provide fixed effects linear gain models 
tested for change across time in the percent of students 
in each of the K-PREP categories. 


The results suggest that use of Kentucky’s mathematics 
innovation in the general education setting improved 
outcomes for students with disabilities and Black 
students. Quality instruction for all is imperative to 
ensure equitable opportunity and access to effective 
instruction for underrepresented populations — students 
cannot benefit from innovations they do not experience 
(Fixsen et al., 2005). Kentucky leaders and educators 
believed if they provided professional learning, technical 
assistance, and support to elementary and middle 
school teachers they would contiguously improve 
teacher practice and improve mathematics outcomes 

for students with disabilities. Fixsen et al. (2013) 
propose effective implementation methods are required 
to ensure consistent use of programs by teachers 

that reliably benefit students. Participating TZ schools 
received ongoing support to develop the capacity to use 
Kentucky’s mathematics innovation and the training, 
coaching, and data use systems designed to effectively 
implement the State Systemic Improvement Plan. The 
onus can no longer rest on the shoulders of teachers and 
school staff. A systematic approach is required to support 
teachers’ growth and development (Darling-Hammond et 
al., 2020). 
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Students with Disabilities 


Figure A shows the percentage of 6th- to 8th-grade 
students with a disability and an Individualized Education 
Program (IEP). The percentage of proficient and 
distinguished math students with disabilities increased in 
TZ schools from 2015-16 to 2018-19 (12.1% to 15.8%). In 
contrast, students with disabilities in the non-TZ schools 
showed a decrease in the proficient and distinguished 
math category (13.8% to 10.4%) during the same school 
years. The difference in percentage change in the 
proficient and distinguished category between TZ and 
non-TZ schools is associated with an effect size of d= 
0.73 standard deviations, a large difference, favoring 

TZ schools. An effect size is a standardized way to 
characterize change. To help interpret the effect size, Hill 
et al. (2008) reported that regular education students in 
Grades 6 to 8 typically improve at a rate of about 0.30 
standard deviations from a full year of mathematics 
instruction in school. The effect size of 0.73, however, 
compares gains among students in TZ schools to 
students in non-TZ schools, all of whom received a full 
year of instruction. 


Novice Apprentice Proficient & Distinguished 


40% 

mnon-TZ wm TZ 
30% 
20% 
” iH § 
0% 


2015-16 2018-19 2015-16 2018-19 2015-16 2018-19 


Figure A. 6th - 8th grade students with a disability and IEP 
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Figure B shows the percentage of students in Grades 3 
to 5 with a disability and an IEP. From 2015-16 to 2018- 
19, the percentage of students with disabilities in the 
novice category decreased in the TZ schools (47%% to 
35%) but increased in non-TZ schools (48% to 49%). 
These students increased in the apprentice category 

in TZ schools (831% to 35%) but decreased in non- 

TZ schools (34% to 31%). They also increased in the 
proficient and distinguished category in TZ schools (21% 
to 30%) more than in non-TZ schools (18% to 20%). 


The three comparisons were associated with effect 
sizes of d = -0.90, 0.46, and 0.62, respectively, all large 
differences. Regular education students in Grades 3 

to 5 typically improve at a rate of about 0.50 standard 
deviations from a full year of school (Hill et al., 2008). 
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Figure B. 3rd - 5th grade students with a disability and IEP 
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Figure C. 6th - 8th grade Black students 


HUNC FRANK PORTER GRAHAM 
E CHILD DEVELOPMENT INSTITUTE 


Students who are Black 


The goal of the State Systemic Improvement Plan was 

to improve mathematics outcomes for students with 
disabilities in the general education setting. Therefore, 

all students had access to Kentucky’s mathematics 
innovation. In TZ schools, this effort improved outcomes 
for students with disabilities and Black students. We report 
the results for Black students to demonstrate the power 

of quality general education instruction when teachers 
continuously improve their skills and engage students in 
meaningful interactions with the content. 


Figure C shows the percentage of 6th- to 8th-grade 
Black students in each math category. One notable result 
emerged. The percent of Black students in the novice 
category decreased in TZ schools from 2015-16 to 2018- 
19 (35.7% to 34.9%) but increased sharply in the non-TZ 
schools (24.1% to 39.6%), a large effect of d = —0.85. 
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Kentucky’s 
Implementation 
Process 


The Kentucky Department of Education (KDE) and the 
SISEP Technical Assistance Center used a stage-based 
approach to intentionally plan and guide all decisions 
regarding implementation (Stages of Implementation: 
httos://nirn.fog.unc.edu/module-4). During exploration, 
they assessed the fit between KDE goals and the intensive 
implementation informed support SISEP provides. They 
identified if there was buy-in from KDE leaders and their 
educational partners and gauged the willingness of state 
and regional leaders to form a team, meet monthly, and 
visibly champion all aspects of the work ahead. Exploration 
activities concluded with a mutual agreement to proceed 
as KDE began installation stage activities. 


State Implementation and Scaling-up Page 9 
of Evidence-based Practices 


Getting from “here” to 
“there” is rarely obvious or 
a straight line. But radical 
imagination — daring to ask 
What if?- is always part of 
the process. 


Villanueva, 2018 


Installation Stage 


During installation, stakeholder teams with diverse 
perspectives, expertise, and roles create the infrastructure 
necessary to effectively implement a selected program 

or practice (Metz, 2015). During the installation stage, 

two regional cooperatives and a large urban district 
mutually agreed to participate in the Transformation Zone. 
The SISEP Center provided monthly onsite and virtual 
implementation informed support. Throughout the process, 
KDE and participating Regional Cooperatives developed 
the capacity to use the Active Implementation Frameworks 
with fidelity to support use of the mathematics innovation 
in the four Transformation Zone districts selected in 2015- 
16. State coaches, known as transformation specialists, 
and the regional coaches supported district teams 
monthly. Coaches used an “I do, we do, you do” process. 
In this process, coaches demonstrate a procedure, guide 
practice, and then provide opportunities for independent 
practice while observing and correcting mistakes (Fisher, 
2008). 


Simultaneously, as implementation teams were 
continuously supported, a diverse team of stakeholders 
(i.e., Institutes of Higher Education and regional, district, 
and school staff) defined and operationalized Kentucky’s 
mathematics innovation. Co-creation of Kentucky’s Usable 
Innovation: A How-To-Guide describes the process and 
provides examples and easy-to-use templates (Ryan 
Jackson et al., 2020). Then, KDE agreed to use the 
Observation Tool for Instructional Supports and Systems 
(OTISS; Fixsen et al., 2020) because a fidelity measure 
for mathematics instruction was not available. The 
OTISS comprises seven items derived from Hattie’s 
(2009) findings on instructional practice that influence 
academic achievement. The OTISS is not used for 
teacher evaluation; instead, it assesses the quality of 
instructional practice and the support systems needed 
to enhance teacher practice. Teachers use OTISS data 
in aggregate to set schoolwide goals for improvement, 
plan lessons, conduct peer-to-peer observations, and 
continuously improve their instructional practice. A blog_ 
post that describes the OTISS (Ryan Jackson, 2020) 
and a series of podcasis with Kentucky educators who 
use the OTISS are available through the SISEP Center 
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When a barrier is 
identified, teams can’t 
keep pushing on, they 
have to stop and solve 
the barrier 


Fixsen, 2019 


a% SISEP 


(Ryan Jackson, Bailey et al., 2020). In the 2016-17 school 
year, the state, Regional Cooperatives, and districts were 
ready to select schools and teachers to participate. With a 
mathematics innovation in place, as well as accompanying 
systems of training, coaching, and data use, the Regional 
Cooperatives began preparing and training district and 
school teams, coaches, and teachers, so they were ready 
to enter the initial implementation stage. 


Initial Implementation 


Teachers begin to use the mathematics innovation and 
teams provide ongoing support to teachers and school 
teams during initial implementation. In the 2016-17 school 
year, initial implementation was initiated in Kentucky in 
Transformation Zone schools. Teachers and schools, 
districts and regions began using Plan-Do-Study-Act 
Cycles (Bryk et al., 2015). While Plan-Do-Study-Act Cycles 
are intuitive to educators, they require considerable time 
and resources to implement effectively (Tichnor-Wagner 
et al., 2017). Therefore, district, school, and teacher 
teams received ongoing training and coaching to analyze 
data, identify a problem and solution, and develop a 

plan to identify the people responsible for actionable 
items with a goal and timeline for completion. Using an 
effective team meeting protocol, teams in Kentucky’s 
Transformation Zone repeat this cycle monthly using 
multiple forms of implementation and outcome data stored 
in the Kentucky Data Dashboard (see a demonstration of 
the Data Dashboard: https://datastudio.google.com/u/0/ 
reporting/545fa6c8-784b-413f-8c9e-5d6b5b7878fe/ 
page/pwccB). Teams identify barriers and report them 

to the team with the resources and authority to deliver 
viable solutions to remove the barriers (Blase, Fixsen, & 
Ryan Jackson, 2015). Kentucky’s implementation teams 
embrace the research that suggests any one level of 
capacity without the other levels is found to be insufficient 
for effective and sustainable change at scale because, 
“when a barrier is identified, teams can’t keep pushing on, 
they have to stop and solve the barrier” Fixsen, 2019, in 
conversation (Fixsen et al., 2018). 
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Improving student 
outcomes requires not 
only engaging the hearts 
and minds of educators 
and stakeholders...but 
also changing the actions 
and behavior patterns of 
teachers, administrators, 
professional development 
providers and policy makers 
— and getting involved in 
systems change. 


Blase et al., 2015 
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During the COVID-19 pandemic implementation was not 
halted. Some regions, districts, and TZ schools continued 
team meetings in virtual contexts. Some collected 
classroom observation data and provided teachers with 
actionable feedback to strengthen equitable access to 
effective instruction by all students. Kentucky continued 
to cast a wide net to engage stakeholders in this process 
because they believe expertise and solutions are found 
close to the ground where communities have direct 
experience of an issue (Villanueva, 2018). Teams work 
together to remove the variability in the system of supports 
available to teachers, from classroom to classroom and 
school to school, to increase equitable opportunity for all 
students to access high quality mathematics instruction 
(Akiba et al., 2007; Harper, 2019; Langley et al., 2009). 


Full Implementation 


Full implementation is achieved when at least 50% or more 
of practitioners are using the innovation with fidelity and 
improved outcomes are achieved. In the 2018-19 school 
year 50% of Kentucky’s Transformation Zone schools and 
teachers moved into full implementation because they 
were using the mathematics innovation with fidelity and 
student mathematics outcomes were improving. The state 
and regional teams continue to take responsibility for all 
implementation efforts as they share lessons learned, 

such as feasibility of implementation, common challenges, 
and successes to be replicated and sustained through 
adequate resourcing and support. They convene monthly 
to use data to critically examine and improve the support 
systems for districts, schools, and teachers using feedback 
received at in person meetings, emails, and surveys. 
Teams use the Kentucky Data Dashboard to assess the 
implementation capacity of teams and they use training, 
coaching, and classroom observation data to assess if 

skill development is transferring to effective application by 
teams and teachers. With a system of supports proven 

to be effective, Kentucky leadership and educators have 
scaled use of the mathematics innovation to additional 
regions, districts, schools, and teachers. 

Blase and colleagues capture the steadfast and persistent 
mission driven approach of Kentucky’s educational leaders 
and educators when they state, “Improving student 
outcomes requires not only engaging the hearts and minds 
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of educators and stakeholders...but also changing the 
actions and behavior patterns of teachers, administrators, 
professional development providers and policy makers — 
and getting involved in systems change” (Blase, Fixsen, 
Sims, & Ward, 2015, p. 4). 


Su mm ary Providing equitable access to effective mathematics 
instruction for all students, and especially for 
underrepresented populations, is a moral imperative 
(Banks, 2007; Fullan, 2010; Harper, 2019). The present 
study demonstrates how one state, in collaboration with 
their educational leaders and educators, was able to close 
long standing disparities in student outcomes for students 
with disabilities and students who are Black in the four 
Transformation Zone districts and 33 schools. Educational 
leaders in Kentucky were willing to acknowledge, “The 
problem is most districts don’t have the time or resources 
to develop all of the systems and measures needed to 
use an innovation effectively” (Ryan Jackson et al., 2021, 
p.5). District leaders and principals provide compelling 
testimonials to their colleagues as they share, “I know we 
are going to build teachers’ capacity, but not only are we 
going to build it, we are going to sustain it” because, “we 
get to say, ‘what is the next best question we can ask?’ and 
try to solve it” (Ryan Jackson et al., 2018, p. 15 & p. 17). 
Kentucky’s educational leaders and educators are focused 
on what is needed. 
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Harper (2019) suggests what is needed is direct 
observation of teacher practice, a standard process to 
analyze data, and adequate time for teacher collaboration 
to increase their confidence in consistently providing 

all students access to high-status knowledge and skills 
(Chubbuck & Zembylas, 2008). The NCTM (2020) policy 
agenda emphasizes that it is teachers and what they 

do in the classroom that produces college-career and 
citizen-ready high school graduates. State and national 
organizations are calling for a strong foundation in 
mathematical literacy and the ability to apply it in pre-K 
through grade 12, for all students, because it is paramount 
to workforce productivity, economic stability, and social 


| know we are going to justice for each and every student (NCTM, 2020). 

build teachers’ capacity, 

but not only are we going Limitations 

to build it, we are going to Although promising results for schools implementing 
sustain it, because we get the State Systemic Improvement Plan and Kentucky’s 

to say, what is the next best mathematics innovation have emerged from the external 
question we can ask and evaluation, there are notable limitations. Schools were not 


randomized to participate; thus, the evaluation represents 
a quasi-experimental design. Although comparable non- 
TZ schools were selected, without randomization it is 
School principal in the possible that TZ schools implementing the State Systemic 
Transformation Zone Improvement Plan differed in meaningful ways from the 
comparison schools and these potential differences could 
have influenced the results. Also, the K-PREP data was 
aggregated at the school-level and did not allow for the 
direct analysis of student-level data nested within schools, 
which could result in lower estimates of the true effects. 


try to solve it. 


Implications 


The research is clear — the quality of teachers’ instructional 
practice in the classroom is the largest factor in teaching 
mathematics for closing long standing deficits and 
disparities in student outcomes (Harper, 2019; Hattie, 
2003; NCTM, 2020). Without a measure of instructional 
practice and follow-up feedback, teachers cannot learn 
and transfer new skills to their instructional practice 
(McKenna & Parenti, 2017), nor can the teams responsible 
for effective implementation understand if there is a lack 
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of standardized processes (Hill & Erickson, 2019) such 
as protocols for teams to use data and tools for decision 
making (March et al., 2016) or adequate and equitable 
resourcing (i.e., ongoing training and coaching based on 
context and need). These strategies are all necessary 

to ascertain whether effectiveness, or lack thereof, is 
attributed to treatment fidelity or other contextual factors 
(Archibald et al., 2011; LeMahieu et al., 2017; Proctor 

et al., 2013). A deeper understanding of the variables 
responsible for the continuous development of educators’ 
effective instruction and implementation capacity will 
further inform the gap between research and practice that 
implementation methods can fill (Allor & Stokes, 2017; 
Fixsen et al, 2013). 


Appen d ix To evaluate the effectiveness of the State Systemic 
Improvement Plan (SSIP) implemented in Kentucky’s TZ 
districts and their schools, an external evaluation team 
conducted a quasi-experimental comparison of 33 TZ 
schools and non-TZ schools. They compared schools on 
the Kentucky Performance Rating for Educational Progress 
(K-PREP) publicly available data. The evaluation team 
compared data from the 2015-16 and 2018-19 school 
years. 


Data analysis 


Fixed-effects linear gain models tested change across 
time in the percent of students in each of the K-PREP 
math categories (novice, apprentice, and proficient and 
distinguished). The category scores from the 2015-2016 
school year were set as the intercept and the 2018-2019 
category scores as the endpoint. The models included 

an intercept (defined as the 2015-2016 math category 
scores), the main effect of Group (coded 1 for TZ schools 
and 0 for non-TZ schools), Time, and the interaction of 
Group x Time. The Group x Time interaction term is a 

test of the effectiveness of the TZ schools implementing 
the State Systemic Improvement Plan to improve student 
math outcomes relative to the comparison non-TZ schools 
under the assumption that other systematic factors had 
not caused those differences. The 33 schools available for 
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the analyses lacked statistical power to detect potentially 
meaningful group differences as statistically significant. 
Thus, Group x Time estimates were reported as d statistics 
(Feingold, 2009), which were used to evaluate the efficacy 
of the TZ schools implementing the State Systemic 
Improvement Plan relative to the matched non-TZ schools. 


Models were run separately for Grades 3 to 5 and Grades 
6 to 8. Within each grade, estimates were obtained 
separately for students with both a disability and an IEP, 
Black students, and all students. Due to limited data, 
separate estimates for Black students could not be 
estimated for 3rd to 5th grade. All models were estimated 
with the SAS software (Version 9.4) using maximum 
likelihood methods. 


Tables 1-3, starting on the next page, provide the K-PREP 
category means and standard deviations for students 

with disabilities and an IEP, for Black students, and for all 
students, respectively. Tables 4-6 provide results from the 
fixed effects gain models for 3rd- to 5th-grade students for 
novice, apprentice, and proficient and distinguished math 
categories, respectively. Tables 7-9 provide results from 
the fixed effects gain models for 6th- to 8th-grade students 
for novice, apprentice, and proficient and distinguished 
math categories, respectively. Table 10 provides a 
summary of all the effect size estimates for the Group x 
Time interactions (Cohen, 1988). 


NATIONAL IMPLEMENTATION 


FRANK PORTER GRAHAM 
QUNC | CHILD DEVELOPMENT INSTITUTE O NIRIN research wetwor Page! 16 


FRANK PORTER GRAHAM CHILD DEVELOPMENT INSTITUTE 


TABLE 1. 


Proficient & 
Distinguished (%) 


Standard Deviations 
for Students with a 2015-2016 2018-2019 2015-2016 2018-2019 2015-2016 | 2018-2019 
Disability and an IEP 


K-PREP Means and Novice (%) Apprentice (%) 


Grades 3-5 

TZ schools 47.3 | 99 | 35.4 | 17.3 [| 31.2 | 149 | 348 | 12.7] 214 | 83 | 29.8 | 12.4 
Non-TZ schools . : : 15.4 17.9 

Grades 6-8 

TZ schools 51.3 | 14.3 | 45.0 | 14.7 | 36.6 | 13.6 | 39.2 | 12.5 712.1 | 10.0 | 15.8 | 14.0 
Non-TZ schools | 48.1 | 12.8 | 41.0 | 16.3 | 38.1 | 11.5 | 48.6 | 14.9] 13.8 | 06.8 | 10.4 | 6.2 


M = mean, SD = standard deviation, TZ = transformation zone. 


TABLE 2. 
. Proficient & 
N a A tice (9 ik 
HEE eaneana ovice (%) pprentice (%) Distinguished (%) 


Standard Deviations | 9915.2016 | 2018-2019 | 2015-2016 | 2018-2019 | 2015-2016 | 2018-2019 
for Black Students 


M SD M M 
Grades 3-5 
TZ schools 40.7 | 22.6 o- == 31.3 | 14.9 -- == 28.0 | 17.0 aaa =" 
Non-TZ schools | 33.3 == 35.4 | 8.1 50.0 =o 38.7 | 13.0 | 16.7 =o 26.0 | 17.5 
Grades 6-8 
TZ schools 35.7 | 12.7 | 34.9 | 14.1 | 41.0 | 84 | 446 | 14.2 | 23.3 | 12.5 | 20.5 | 11.7 


Non-TZ schools | 24.1 | 16.2 | 39.6 | 19.7 | 41.9 | 10.7 | 41.0 | 12.8 | 34.0 | 16.6 | 19.5 | 11.1 


M = mean, SD = standard deviation, TZ = transformation zone. 


a% SISEP 


A State Implementation and Scaling-up 
of Evidence-based Practices 


Page! 17 


TABLE 3. 


. . Proficient & 
° fe) 

K-PREP Means and Novice (%) Apprentice (%) Distinguished (%) 
Standard 
Geviationsder All 2015-2016 | 2018-2019 | 2015-2016 | 2018-2019 | 2015-2016 | 2018-2019 
students SD SD M SD SD 
Grades 3-5 
TZ schools 23.6 pe 21.6 Le 30.6 7.0 33.1 6.7 45.8 9.5 45.3 | 12.2 


Non-TZ schools | 14.7 | 9.2 | 16.4) 7.9 | 30.6 | 9.2 | 29.2 | 7.7 | 54.7 | 15.6 | 54.4 | 13.8 


Grades 6-8 


TZ schools 21.9 |O7) 20.3 (Pei) 36.5 eg) 39.5 | 11.4 | 41.9 tay) 40.3 io 


Non-TZ schools | 12.9 | 8.1 13.9 [Bi24a] 31.9 (B25) 35.7 | 7.9 | 55.2 tease 50.4 ie 


M = mean, SD = standard deviation, TZ = transformation zone. 


TABLE 4. 


Results of Fixed Effects Gain Models 
for 3rd-5th Grade Novice in Math 


Estimate SE t-value p-value 


Students with Disability & IEP 


Intercept 483 .030 16.06 <.001 
Group 017 .066 0.25 .804 
Slope .006 .033 0.18 856 
Group x time -.133 .068 -1.94 .056 
All students 
Intercept 147 .018 8.19 <.001 
Group .108 .040 2.67 .017 
Slope .017 .013 1.31 .193 
Group x time -.041 .027 -1.55 i225) 


SE = standard error. The group x time estimate for all students associated with Cohen’s d = -.47 and for 
students with a disability d = -.90. 
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TABLE 5. 


Results of Fixed Effects Gain Models for 
3rd-5th Grade Apprentice in Math 


Estimate SE t-value p-value 


Students with Disability & IEP 


Intercept 339 022 15.20 <.001 
Group -.036 047 -0.76 .456 
Slope -.025 .027 -0.91 .367 
Group x time .064 .056 1.14 .260 
All students 
Intercept .306 .017 17.51 <.001 
Group 026 .039 0.67 15 
Slope -.014 .013 -1.06 .291 
Group x time .035 .027 1.32 191 


SE = standard error. The group x time estimate for all students associated with Cohen’s d = .40 and for 


students with a disability d = .46. 


TABLE 6. 


Results of Fixed Effects Gain Models for 
3rd-5th Grade Proficient & Distinguished in 
Math 


Students with Disability & IEP 


Estimate 


SE 


t-value 


p-value 


Intercept 181 023 7.72 <.001 
Group .027 .052 0.52 .611 
Slope .017 .024 0.71 479 
Group x time .067 .050 1.36 nee 
All students 
Intercept 547 033 16.52 <.001 
Group -.144 .073 -1.97 .067 
Slope -.003 .018 -0.18 853 
Group x time .009 .038 0.24 .809 


SE = standard error. The group x time estimate for all students associated with Cohen’s d = .06 and for 


students with a disability d = 62. 
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TABLE 7. 


Results of Fixed Effects Gain Models for 
6th-8th Grade Novice in Math 


Estimate 


SE 


t-value 


p-value 


Students with Disability & IEP 


Intercept 481 051 9.48 <.001 
Group .044 .060 0.73 477 
Slope -.071 .039 -1.82 .072 
Group x time .017 .046 0.36 122 
Students who are Black 
Intercept .234 .057 4.14 .002 
Group 129 .068 1.91 .083 
Slope 113 .056 2.04 .046 
Group x time -.116 .063 -1.84 .071 
All students 
Intercept .129 041 3.09 .008 
Group 102 .050 2.05 .058 
Slope .010 016 0.60 551 
Group x time -.020 019 -1.05 .298 


SE = standard error. The group x time estimate for all students associated with Cohen’s d = -.21, for 


students who are Black d = -.85, and for students with a disability d = 12. 
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TABLE 8. 


Results of Fixed Effects Gain Models for 
6th-8th Grade Apprentice in Math 


Students with Disability & IEP 


Estimate SE 


t-value 


p-value 


Intercept 381 041 9.32 <.001 
Group -.010 .049 -0.21 sees) 
Slope .105 .042 2.52 .014 
Group x time -.082 .049 -1.66 101 
Students who are Black 
Intercept 419 034 12.32 <.001 
Group -.009 .040 -0.23 .820 
Slope -.009 054 -0.17 .863 
Group x time 046 .062 0.73 466 
All students 
Intercept 319 035 9.20 <.001 
Group .055 041 1.34 199 
Slope .039 025 1:55 25 
Group x time -.006 029 -0.20 842 


SE = standard error. The group x time estimate for all students associated with Cohen’s d = -07, for 
Students who are Black d = .51, and for students with a disability d = -.63. 
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TABLE 9. 


Results of Fixed Effects Gain Models for 
6th-8th Grade Proficient & Distinguished in 
Math 


Students with Disability & IEP 


Estimate 


SE 


t-value 


p-value 


Students who are Black 


Intercept 138 032 4.25 <.001 
Group -.031 .038 -0.81 431 
Slope -.033 .033 -1.22 ron 
Group x time .067 .039 1.71 .091 


Intercept 349 .050 7.01 <.001 
Group -.126 .060 -2.11 .059 
Slope -.105 .050 -2.10 .040 
Group x time .074 .057 1.30 199 
All students 
Intercept 552 .067 8.35 <.001 
Group -.159 .079 -2.02 .062 
Slope -.048 .027 -1.82 .072 
Group x time .026 .031 0.82 412 


SE = standard error. The group x time estimate for all students associated with Cohen’s d = .17, for 
students who are Black d = 54, and for students with a disability d = .73. 
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TABLE 10. 


Effect Size Estimates (Cohen’s d) for Group x Time 
Interactions. 


Grades 3-5 


SE 


Proficient and Distinguished 


All students .06 sllZ 
Students who are Black -- 54 
Students with a disability and IEP .62 13 


a% SISEP 


AN State Implementation and Scaling-up 


of Evidence-based Practices 


Apprentice 
All students .40 -.07 
Students who are Black - 51 
Students with a disability and IEP 46 -.63 
Novice 
All students -.47 -.21 
Students who are Black - -.85 
Students with a disability and IEP -.90 12 
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